SUPERAR - reanalysis of meta-analyses on the effect of music education
Ivan Ropovik
2023-12-18
knitr::opts_chunk$set(echo = FALSE, warning = FALSE)This is the supplementary analytic output for the reanalysis of meta-analyses on the effect of music education, carried out for the SUPERAR project
Brief information about the methods used in the analysis:
RMA results with model-based SEs k = number of studies; sqrt in “Variance components” = tau, the standard deviation of true effects; estimate in “Model results” = naive MA estimate
RVE SEs with Satterthwaite small-sample correction Estimate based on a multilevel RE model with constant sampling correlation model (CHE - correlated hierarchical effects - working model) (Pustejovsky & Tipton, 2020; https://osf.io/preprints/metaarxiv/vyfcj/). Interpretation of naive-meta-analysis should be based on these estimates.
Prediction interval Shows the expected range of true effects in similar studies. As an approximation, in 95% of cases the true effect in a new published study can be expected to fall between PI LB and PI UB. Note that these are non-adjusted estimates. An unbiased newly conducted study will more likely fall in an interval centered around bias-adjusted estimate with a wider CI width.
Heterogeneity Tau can be interpreted as the total amount of heterogeneity in the true effects. I^2$ represents the ratio of true heterogeneity to total variance across the observed effect estimates. Estimates calculated by two approaches are reported. This is followed by separate estimates of between- and within-cluster heterogeneity and estimated intra-class correlation of underlying true effects.
Proportion of significant results What proportion of effects were statistically at the alpha level of .05.
ES-precision correlation Kendalls’s correlation between the ES and precision.
4/3PSM Applies a permutation-based, step-function 4-parameter selection model (one-tailed p-value steps = c(.025, .5, 1)). Falls back to 3-parameter selection model if at least one of the three p-value intervals contains less than 5 p-values. For this meta-analysis, we applied 3-parameter selection model by default as there were only 11 independent effects in the opposite direction overall (6%), causing the estimates to be unstable across iterations. pvalue = p-value testing H0 that the effect is zero. ciLB and ciUB are lower and upper bound of the CI. k = number of studies. steps = 3 means that the 4PSM was applied, 2 means that the 3PSM was applied. We also ran two sensitivity analyses of the selection model, the Vevea & Woods (2005) step function model with a priori defined selection weights and the Robust Bayesian Meta-analysis model employing the model-averaging approach (Bartoš & Maier, 2020).
PET-PEESE Estimated effect size of an infinitely precise study. Using 4/3PSM as the conditional estimator instead of PET (can be changed to PET). If the PET-PEESE estimate is in the opposite direction, the effect can be regarded nil. By default (can be changed to PET), the function employs a modified sample-size based estimator (see https://www.jepusto.com/pet-peese-performance/). It also uses the same RVE sandwich-type based estimator in a CHE (correlated hierarchical effects) working model with the identical random effects structure as the primary (naive) meta-analytic model.
We report results for both, PET and PEESE, with the first reported one being the primary (based on the conditional estimator).
WAAP-WLS The combined WAAP-WLS estimator (weighted average of the adequately powered - weighted least squares) tries to identify studies that are adequately powered to detect the meta-analytic effect. If there is less than two such studies, the method falls back to the WLS estimator (Stanley & Doucouliagos, 2015). If there are at least two adequately powered studies, WAAP returns a WLS estimate based on effects from only those studies.
type = 1: WAAP estimate, 2: WLS estimate. kAdequate = number of adequately powered studies
p-uniform P-uniform* is a selection model conceptually similar to p-curve. It makes use of the fact that p-values follow a uniform distribution at the true effect size while it includes also nonsignificant effect sizes. Permutation-based version of p-uniform method, the so-called p-uniform* (van Aert, van Assen, 2021).
p-curve Permutation-based p-curve method. Output should be self-explanatory. For more info see p-curve.com
Power for detecting SESOI and bias-corrected parameter estimates Estimates of the statistical power for detecting a smallest effect sizes of interest equal to .20, .50, and .70 in SD units (Cohen’s d). A sort of a thought experiment, we also assumed that population true values equal the bias-corrected estimates (4/3PSM or PET-PEESE) and computed power for those.
Handling of dependencies in bias-correction methods To handle dependencies among the effects, the 4PSM, p-curve, p-uniform are implemented using a permutation-based procedure, randomly selecting only one focal effect (i.e., excluding those which were not coded as being focal) from a single study and iterating nIterations times. Lastly, the procedure selects the result with the median value of the ES estimate (4PSM, p-uniform) or median z-score of the full p-curve (p-curve).
Results
## $`1`
## $`1`$spatial
## $`1`$spatial$resultsOverall
## $`1`$spatial$resultsOverall$`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 19; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 8 no paperID
## sigma^2.2 0.0481 0.2193 19 no paperID/effectID
##
## Test for Heterogeneity:
## Q(df = 18) = 28.4390, p-val = 0.0557
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.2019 0.1264 1.5968 0.1103 -0.0459 0.4497
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`1`$spatial$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`
## $`1`$spatial$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## intrcpt 0.202 0.121 1.67 6.16 0.146
##
## $`1`$spatial$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef. Estimate SE d.f. Lower 95% CI Upper 95% CI
## intrcpt 0.202 0.121 6.16 -0.0928 0.497
##
##
## $`1`$spatial$resultsOverall$`Prediction interval`
## 95% PI LB 95% PI UB
## -0.389 0.793
##
## $`1`$spatial$resultsOverall$Heterogeneity
## Tau I^2
## 0.2193068 25.5078742
## Jackson's I^2 Between-cluster heterogeneity
## 52.3300000 0.0000000
## Within-cluster heterogeneity ICC
## 25.5100000 0.0000000
##
## $`1`$spatial$resultsOverall$`Proportion of significant results`
## [1] 0.05263158
##
## $`1`$spatial$resultsOverall$`Publication bias`
## [1] "Publication bias corrections not carried out"
##
##
## $`1`$spatial$resultsTable
## k g [95% CI] 95% PI [LB, UB] SE
## 19 0.2 [-0.09, 0.5] [-0.389, 0.793] 0.12
## Tau I^2
## 0.22 26%
##
##
## $`1`$processingSpeed
## $`1`$processingSpeed$resultsOverall
## $`1`$processingSpeed$resultsOverall$`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 22; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0272 0.1650 8 no paperID
## sigma^2.2 0.0854 0.2922 22 no paperID/effectID
##
## Test for Heterogeneity:
## Q(df = 21) = 57.4626, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.1726 0.1271 1.3576 0.1746 -0.0766 0.4217
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`1`$processingSpeed$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`
## $`1`$processingSpeed$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## intrcpt 0.173 0.128 1.35 6.68 0.221
##
## $`1`$processingSpeed$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef. Estimate SE d.f. Lower 95% CI Upper 95% CI
## intrcpt 0.173 0.128 6.68 -0.133 0.478
##
##
## $`1`$processingSpeed$resultsOverall$`Prediction interval`
## 95% PI LB 95% PI UB
## -0.676 1.021
##
## $`1`$processingSpeed$resultsOverall$Heterogeneity
## Tau I^2
## 0.3355215 66.0278236
## Jackson's I^2 Between-cluster heterogeneity
## 90.2500000 15.9600000
## Within-cluster heterogeneity ICC
## 50.0600000 0.2400000
##
## $`1`$processingSpeed$resultsOverall$`Proportion of significant results`
## [1] 0.2272727
##
## $`1`$processingSpeed$resultsOverall$`Publication bias`
## [1] "Publication bias corrections not carried out"
##
##
## $`1`$processingSpeed$resultsTable
## k g [95% CI] 95% PI [LB, UB] SE
## 22 0.17 [-0.13, 0.48] [-0.676, 1.021] 0.13
## Tau I^2
## 0.34 66%
##
##
## $`1`$phonolProcessing
## $`1`$phonolProcessing$resultsOverall
## $`1`$phonolProcessing$resultsOverall$`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 39; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 15 no paperID
## sigma^2.2 0.0468 0.2163 39 no paperID/effectID
##
## Test for Heterogeneity:
## Q(df = 38) = 63.5002, p-val = 0.0059
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.1368 0.0814 1.6802 0.0929 -0.0228 0.2963 .
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`1`$phonolProcessing$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`
## $`1`$phonolProcessing$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## intrcpt 0.137 0.0677 2.02 7.76 0.0792 .
##
## $`1`$phonolProcessing$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef. Estimate SE d.f. Lower 95% CI Upper 95% CI
## intrcpt 0.137 0.0677 7.76 -0.0203 0.294
##
##
## $`1`$phonolProcessing$resultsOverall$`Prediction interval`
## 95% PI LB 95% PI UB
## -0.349 0.623
##
## $`1`$phonolProcessing$resultsOverall$Heterogeneity
## Tau I^2
## 0.2162839 34.2520660
## Jackson's I^2 Between-cluster heterogeneity
## 24.0200000 0.0000000
## Within-cluster heterogeneity ICC
## 34.2500000 0.0000000
##
## $`1`$phonolProcessing$resultsOverall$`Proportion of significant results`
## [1] 0.1025641
##
## $`1`$phonolProcessing$resultsOverall$`Publication bias`
## [1] "Publication bias corrections not carried out"
##
##
## $`1`$phonolProcessing$resultsTable
## k g [95% CI] 95% PI [LB, UB] SE
## 39 0.14 [-0.02, 0.29] [-0.349, 0.623] 0.07
## Tau I^2
## 0.22 34%
##
##
## $`1`$memory
## $`1`$memory$resultsOverall
## $`1`$memory$resultsOverall$`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 57; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0133 0.1155 19 no paperID
## sigma^2.2 0.0448 0.2116 57 no paperID/effectID
##
## Test for Heterogeneity:
## Q(df = 56) = 111.3234, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.2531 0.0734 3.4471 0.0006 0.1092 0.3970 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`1`$memory$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`
## $`1`$memory$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## intrcpt 0.253 0.0746 3.39 13.4 0.00466 **
##
## $`1`$memory$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef. Estimate SE d.f. Lower 95% CI Upper 95% CI
## intrcpt 0.253 0.0746 13.4 0.0923 0.414
##
##
## $`1`$memory$resultsOverall$`Prediction interval`
## 95% PI LB 95% PI UB
## -0.276 0.782
##
## $`1`$memory$resultsOverall$Heterogeneity
## Tau I^2
## 0.2410852 54.7441424
## Jackson's I^2 Between-cluster heterogeneity
## 87.2200000 12.5600000
## Within-cluster heterogeneity ICC
## 42.1900000 0.2300000
##
## $`1`$memory$resultsOverall$`Proportion of significant results`
## [1] 0.1754386
##
## $`1`$memory$resultsOverall$`Publication bias`
## [1] "Publication bias corrections not carried out"
##
##
## $`1`$memory$resultsTable
## k g [95% CI] 95% PI [LB, UB] SE
## 57 0.25 [0.09, 0.41] [-0.276, 0.782] 0.07
## Tau I^2
## 0.24 55%
##
##
## $`1`$math
## $`1`$math$resultsOverall
## $`1`$math$resultsOverall$`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 18; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0351 0.1872 9 no paperID
## sigma^2.2 0.0000 0.0000 18 no paperID/effectID
##
## Test for Heterogeneity:
## Q(df = 17) = 17.9298, p-val = 0.3933
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.1940 0.0995 1.9493 0.0513 -0.0011 0.3891 .
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`1`$math$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`
## $`1`$math$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## intrcpt 0.194 0.102 1.9 6.09 0.105
##
## $`1`$math$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef. Estimate SE d.f. Lower 95% CI Upper 95% CI
## intrcpt 0.194 0.102 6.09 -0.0545 0.443
##
##
## $`1`$math$resultsOverall$`Prediction interval`
## 95% PI LB 95% PI UB
## -0.293 0.681
##
## $`1`$math$resultsOverall$Heterogeneity
## Tau I^2
## 0.1872316 44.4943199
## Jackson's I^2 Between-cluster heterogeneity
## 75.9200000 44.4900000
## Within-cluster heterogeneity ICC
## 0.0000000 1.0000000
##
## $`1`$math$resultsOverall$`Proportion of significant results`
## [1] 0.1111111
##
## $`1`$math$resultsOverall$`Publication bias`
## [1] "Publication bias corrections not carried out"
##
##
## $`1`$math$resultsTable
## k g [95% CI] 95% PI [LB, UB] SE
## 18 0.19 [-0.05, 0.44] [-0.293, 0.681] 0.1
## Tau I^2
## 0.19 44%
##
##
## $`1`$literacy
## $`1`$literacy$resultsOverall
## $`1`$literacy$resultsOverall$`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 50; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 23 no paperID
## sigma^2.2 0.0000 0.0000 50 no paperID/effectID
##
## Test for Heterogeneity:
## Q(df = 49) = 42.8907, p-val = 0.7179
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.0600 0.0445 1.3484 0.1775 -0.0272 0.1472
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`1`$literacy$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`
## $`1`$literacy$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## intrcpt 0.06 0.0411 1.46 5.68 0.197
##
## $`1`$literacy$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef. Estimate SE d.f. Lower 95% CI Upper 95% CI
## intrcpt 0.06 0.0411 5.68 -0.0419 0.162
##
##
## $`1`$literacy$resultsOverall$`Prediction interval`
## 95% PI LB 95% PI UB
## -0.02 0.14
##
## $`1`$literacy$resultsOverall$Heterogeneity
## Tau I^2
## 5.245164e-06 3.909034e-08
## Jackson's I^2 Between-cluster heterogeneity
## 0.000000e+00 0.000000e+00
## Within-cluster heterogeneity ICC
## 0.000000e+00 8.800000e-01
##
## $`1`$literacy$resultsOverall$`Proportion of significant results`
## [1] 0.04
##
## $`1`$literacy$resultsOverall$`Publication bias`
## [1] "Publication bias corrections not carried out"
##
##
## $`1`$literacy$resultsTable
## k g [95% CI] 95% PI [LB, UB] SE
## 50 0.06 [-0.04, 0.16] [-0.02, 0.14] 0.04
## Tau I^2
## 0 0%
##
##
## $`1`$intelligence
## $`1`$intelligence$resultsOverall
## $`1`$intelligence$resultsOverall$`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 32; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 16 no paperID
## sigma^2.2 0.0171 0.1309 32 no paperID/effectID
##
## Test for Heterogeneity:
## Q(df = 31) = 36.1729, p-val = 0.2397
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.1133 0.0735 1.5409 0.1233 -0.0308 0.2574
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`1`$intelligence$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`
## $`1`$intelligence$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## intrcpt 0.113 0.0669 1.69 7.52 0.131
##
## $`1`$intelligence$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef. Estimate SE d.f. Lower 95% CI Upper 95% CI
## intrcpt 0.113 0.0669 7.52 -0.0427 0.269
##
##
## $`1`$intelligence$resultsOverall$`Prediction interval`
## 95% PI LB 95% PI UB
## -0.197 0.423
##
## $`1`$intelligence$resultsOverall$Heterogeneity
## Tau I^2
## 0.1308709 18.6747020
## Jackson's I^2 Between-cluster heterogeneity
## 64.5900000 0.0000000
## Within-cluster heterogeneity ICC
## 18.6700000 0.0000000
##
## $`1`$intelligence$resultsOverall$`Proportion of significant results`
## [1] 0.09375
##
## $`1`$intelligence$resultsOverall$`Publication bias`
## [1] "Publication bias corrections not carried out"
##
##
## $`1`$intelligence$resultsTable
## k g [95% CI] 95% PI [LB, UB] SE
## 32 0.11 [-0.04, 0.27] [-0.197, 0.423] 0.07
## Tau I^2
## 0.13 19%
##
##
## $`1`$inhibition
## $`1`$inhibition$resultsOverall
## $`1`$inhibition$resultsOverall$`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 17; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 7 no paperID
## sigma^2.2 0.0528 0.2297 17 no paperID/effectID
##
## Test for Heterogeneity:
## Q(df = 16) = 32.9813, p-val = 0.0074
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.0557 0.1120 0.4969 0.6193 -0.1639 0.2752
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`1`$inhibition$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`
## $`1`$inhibition$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## intrcpt 0.0557 0.0764 0.728 3.68 0.51
##
## $`1`$inhibition$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef. Estimate SE d.f. Lower 95% CI Upper 95% CI
## intrcpt 0.0557 0.0764 3.68 -0.164 0.275
##
##
## $`1`$inhibition$resultsOverall$`Prediction interval`
## 95% PI LB 95% PI UB
## -0.535 0.647
##
## $`1`$inhibition$resultsOverall$Heterogeneity
## Tau I^2
## 0.2297003 42.0317851
## Jackson's I^2 Between-cluster heterogeneity
## 37.4200000 0.0000000
## Within-cluster heterogeneity ICC
## 42.0300000 0.0000000
##
## $`1`$inhibition$resultsOverall$`Proportion of significant results`
## [1] 0.1176471
##
## $`1`$inhibition$resultsOverall$`Publication bias`
## [1] "Publication bias corrections not carried out"
##
##
## $`1`$inhibition$resultsTable
## k g [95% CI] 95% PI [LB, UB] SE
## 17 0.06 [-0.16, 0.28] [-0.535, 0.647] 0.08
## Tau I^2
## 0.23 42%
##
##
##
## $`2`
## $`2`$phonologicalAwareness
## $`2`$phonologicalAwareness$resultsOverall
## $`2`$phonologicalAwareness$resultsOverall$`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 18; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 11 no paperID
## sigma^2.2 0.0699 0.2643 18 no paperID/effectID
##
## Test for Heterogeneity:
## Q(df = 17) = 36.1512, p-val = 0.0044
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.2177 0.1015 2.1447 0.0320 0.0187 0.4166 *
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`2`$phonologicalAwareness$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`
## $`2`$phonologicalAwareness$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## intrcpt 0.218 0.0934 2.33 8.27 0.0472 *
##
## $`2`$phonologicalAwareness$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef. Estimate SE d.f. Lower 95% CI Upper 95% CI
## intrcpt 0.218 0.0934 8.27 0.00344 0.432
##
##
## $`2`$phonologicalAwareness$resultsOverall$`Prediction interval`
## 95% PI LB 95% PI UB
## -0.405 0.840
##
## $`2`$phonologicalAwareness$resultsOverall$Heterogeneity
## Tau I^2
## 0.2643395 51.8018030
## Jackson's I^2 Between-cluster heterogeneity
## 54.7300000 0.0000000
## Within-cluster heterogeneity ICC
## 51.8000000 0.0000000
##
## $`2`$phonologicalAwareness$resultsOverall$`Proportion of significant results`
## [1] 0.1111111
##
## $`2`$phonologicalAwareness$resultsOverall$`Publication bias`
## [1] "Publication bias corrections not carried out"
##
##
## $`2`$phonologicalAwareness$resultsTable
## k g [95% CI] 95% PI [LB, UB] SE Tau
## 18 0.22 [0, 0.43] [-0.405, 0.84] 0.09 0.26
## I^2
## 52%
##
##
## $`2`$readingFLuency
## $`2`$readingFLuency$resultsOverall
## $`2`$readingFLuency$resultsOverall$`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 5; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 5 no paperID
## sigma^2.2 0.0000 0.0000 5 no paperID/effectID
##
## Test for Heterogeneity:
## Q(df = 4) = 6.2812, p-val = 0.1791
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.1579 0.0966 1.6338 0.1023 -0.0315 0.3473
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`2`$readingFLuency$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`
## $`2`$readingFLuency$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## intrcpt 0.158 0.0772 2.05 1.99 0.178
##
## $`2`$readingFLuency$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef. Estimate SE d.f. Lower 95% CI Upper 95% CI
## intrcpt 0.158 0.0772 1.99 -0.176 0.492
##
##
## $`2`$readingFLuency$resultsOverall$`Prediction interval`
## 95% PI LB 95% PI UB
## -0.056 0.371
##
## $`2`$readingFLuency$resultsOverall$Heterogeneity
## Tau I^2
## 4.629868e-06 3.576893e-08
## Jackson's I^2 Between-cluster heterogeneity
## 0.000000e+00 0.000000e+00
## Within-cluster heterogeneity ICC
## 0.000000e+00 5.000000e-01
##
## $`2`$readingFLuency$resultsOverall$`Proportion of significant results`
## [1] 0.2
##
## $`2`$readingFLuency$resultsOverall$`Publication bias`
## [1] "Publication bias corrections not carried out"
##
##
## $`2`$readingFLuency$resultsTable
## k g [95% CI] 95% PI [LB, UB] SE
## 5 0.16 [-0.18, 0.49] [-0.056, 0.371] 0.08
## Tau I^2
## 0 0%
##
##
##
## $`3`
## $`3`$executiveFunctions
## $`3`$executiveFunctions$resultsOverall
## $`3`$executiveFunctions$resultsOverall$`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 24; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 7 no paperID
## sigma^2.2 0.1645 0.4055 24 no paperID/effectID
##
## Test for Heterogeneity:
## Q(df = 23) = 96.4425, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.5118 0.1309 3.9085 <.0001 0.2552 0.7685 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`3`$executiveFunctions$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`
## $`3`$executiveFunctions$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## intrcpt 0.512 0.0958 5.34 4.64 0.00386 **
##
## $`3`$executiveFunctions$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef. Estimate SE d.f. Lower 95% CI Upper 95% CI
## intrcpt 0.512 0.0958 4.64 0.26 0.764
##
##
## $`3`$executiveFunctions$resultsOverall$`Prediction interval`
## 95% PI LB 95% PI UB
## -0.506 1.529
##
## $`3`$executiveFunctions$resultsOverall$Heterogeneity
## Tau I^2
## 0.405533 64.699082
## Jackson's I^2 Between-cluster heterogeneity
## 63.460000 0.000000
## Within-cluster heterogeneity ICC
## 64.700000 0.000000
##
## $`3`$executiveFunctions$resultsOverall$`Proportion of significant results`
## [1] 0.5
##
## $`3`$executiveFunctions$resultsOverall$`Publication bias`
## [1] "Publication bias corrections not carried out"
##
##
## $`3`$executiveFunctions$resultsTable
## k g [95% CI] 95% PI [LB, UB] SE
## 24 0.51 [0.26, 0.76] [-0.506, 1.529] 0.1
## Tau I^2
## 0.41 65%
##
##
##
## $`4`
## $`4`$aggressiveBehavior
## $`4`$aggressiveBehavior$resultsOverall
## $`4`$aggressiveBehavior$resultsOverall$`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 7; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 1.6671 1.2911 4 no paperID
## sigma^2.2 0.0000 0.0000 7 no paperID/effectID
##
## Test for Heterogeneity:
## Q(df = 6) = 77.7832, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## -1.2086 0.6760 -1.7878 0.0738 -2.5336 0.1164 .
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`4`$aggressiveBehavior$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`
## $`4`$aggressiveBehavior$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## intrcpt -1.21 0.68 -1.78 2.99 0.174
##
## $`4`$aggressiveBehavior$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef. Estimate SE d.f. Lower 95% CI Upper 95% CI
## intrcpt -1.21 0.68 2.99 -3.38 0.96
##
##
## $`4`$aggressiveBehavior$resultsOverall$`Prediction interval`
## 95% PI LB 95% PI UB
## -5.846 3.429
##
## $`4`$aggressiveBehavior$resultsOverall$Heterogeneity
## Tau I^2
## 1.291144 99.290049
## Jackson's I^2 Between-cluster heterogeneity
## 99.780000 99.290000
## Within-cluster heterogeneity ICC
## 0.000000 1.000000
##
## $`4`$aggressiveBehavior$resultsOverall$`Proportion of significant results`
## [1] 0.4285714
##
## $`4`$aggressiveBehavior$resultsOverall$`Publication bias`
## [1] "Publication bias corrections not carried out"
##
##
## $`4`$aggressiveBehavior$resultsTable
## k g [95% CI] 95% PI [LB, UB] SE
## 7 -1.21 [-3.38, 0.96] [-5.846, 3.429] 0.68
## Tau I^2
## 1.29 99%
##
##
## $`4`$hyperactivityImpulsivity
## $`4`$hyperactivityImpulsivity$resultsOverall
## $`4`$hyperactivityImpulsivity$resultsOverall$`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 7; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 3 no paperID
## sigma^2.2 1.2423 1.1146 7 no paperID/effectID
##
## Test for Heterogeneity:
## Q(df = 6) = 60.4069, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.0089 0.5003 0.0178 0.9858 -0.9717 0.9895
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`4`$hyperactivityImpulsivity$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`
## $`4`$hyperactivityImpulsivity$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## intrcpt 0.00889 0.263 0.0339 1.7 0.977
##
## $`4`$hyperactivityImpulsivity$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef. Estimate SE d.f. Lower 95% CI Upper 95% CI
## intrcpt 0.00889 0.263 1.7 -1.34 1.36
##
##
## $`4`$hyperactivityImpulsivity$resultsOverall$`Prediction interval`
## 95% PI LB 95% PI UB
## -4.929 4.947
##
## $`4`$hyperactivityImpulsivity$resultsOverall$Heterogeneity
## Tau I^2
## 1.114597 90.604306
## Jackson's I^2 Between-cluster heterogeneity
## 91.840000 0.000000
## Within-cluster heterogeneity ICC
## 90.600000 0.000000
##
## $`4`$hyperactivityImpulsivity$resultsOverall$`Proportion of significant results`
## [1] 0.5714286
##
## $`4`$hyperactivityImpulsivity$resultsOverall$`Publication bias`
## [1] "Publication bias corrections not carried out"
##
##
## $`4`$hyperactivityImpulsivity$resultsTable
## k g [95% CI] 95% PI [LB, UB] SE
## 7 0.01 [-1.34, 1.36] [-4.929, 4.947] 0.26
## Tau I^2
## 1.11 91%
##
##
## $`4`$`self-control`
## $`4`$`self-control`$resultsOverall
## $`4`$`self-control`$resultsOverall$`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 4; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 1.4959 1.2231 3 no paperID
## sigma^2.2 0.0004 0.0203 4 no paperID/effectID
##
## Test for Heterogeneity:
## Q(df = 3) = 86.4838, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.8654 0.7147 1.2109 0.2259 -0.5354 2.2661
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`4`$`self-control`$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`
## $`4`$`self-control`$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## intrcpt 0.865 0.713 1.21 2 0.349
##
## $`4`$`self-control`$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef. Estimate SE d.f. Lower 95% CI Upper 95% CI
## intrcpt 0.865 0.713 2 -2.21 3.94
##
##
## $`4`$`self-control`$resultsOverall$`Prediction interval`
## 95% PI LB 95% PI UB
## -5.230 6.961
##
## $`4`$`self-control`$resultsOverall$Heterogeneity
## Tau I^2
## 1.223253 99.693947
## Jackson's I^2 Between-cluster heterogeneity
## 99.840000 99.670000
## Within-cluster heterogeneity ICC
## 0.030000 1.000000
##
## $`4`$`self-control`$resultsOverall$`Proportion of significant results`
## [1] 0.5
##
## $`4`$`self-control`$resultsOverall$`Publication bias`
## [1] "Publication bias corrections not carried out"
##
##
## $`4`$`self-control`$resultsTable
## k g [95% CI] 95% PI [LB, UB] SE
## 4 0.87 [-2.21, 3.94] [-5.23, 6.961] 0.71
## Tau I^2
## 1.22 100%
##
##
## $`4`$prosocialBehavior
## $`4`$prosocialBehavior$resultsOverall
## $`4`$prosocialBehavior$resultsOverall$`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 4; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 3 no paperID
## sigma^2.2 0.0021 0.0454 4 no paperID/effectID
##
## Test for Heterogeneity:
## Q(df = 3) = 4.8858, p-val = 0.1804
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.0285 0.0472 0.6039 0.5459 -0.0640 0.1210
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`4`$prosocialBehavior$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`
## $`4`$prosocialBehavior$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## intrcpt 0.0285 0.0288 0.991 1.1 0.491
##
## $`4`$prosocialBehavior$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef. Estimate SE d.f. Lower 95% CI Upper 95% CI
## intrcpt 0.0285 0.0288 1.1 -0.269 0.326
##
##
## $`4`$prosocialBehavior$resultsOverall$`Prediction interval`
## 95% PI LB 95% PI UB
## -0.180 0.237
##
## $`4`$prosocialBehavior$resultsOverall$Heterogeneity
## Tau I^2
## 0.04538964 28.89708215
## Jackson's I^2 Between-cluster heterogeneity
## 36.00000000 0.00000000
## Within-cluster heterogeneity ICC
## 28.90000000 0.00000000
##
## $`4`$prosocialBehavior$resultsOverall$`Proportion of significant results`
## [1] NA
##
## $`4`$prosocialBehavior$resultsOverall$`Publication bias`
## [1] "Publication bias corrections not carried out"
##
##
## $`4`$prosocialBehavior$resultsTable
## k g [95% CI] 95% PI [LB, UB] SE
## 4 0.03 [-0.27, 0.33] [-0.18, 0.237] 0.03
## Tau I^2
## 0.05 29%
##
##
## $`4`$aggressionPropensity
## $`4`$aggressionPropensity$resultsOverall
## $`4`$aggressionPropensity$resultsOverall$`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 2; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0017 0.0409 2 no paperID
## sigma^2.2 0.0017 0.0409 2 no paperID/effectID
##
## Test for Heterogeneity:
## Q(df = 1) = 1.0804, p-val = 0.2986
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## -0.0267 0.0688 -0.3883 0.6978 -0.1616 0.1082
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`4`$aggressionPropensity$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`
## $`4`$aggressionPropensity$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## intrcpt -0.0267 0.0688 -0.388 1 0.764
##
## $`4`$aggressionPropensity$resultsOverall$`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef. Estimate SE d.f. Lower 95% CI Upper 95% CI
## intrcpt -0.0267 0.0688 1 -0.901 0.848
##
##
## $`4`$aggressionPropensity$resultsOverall$`Prediction interval`
## 95% PI LB 95% PI UB
## -0.865 0.811
##
## $`4`$aggressionPropensity$resultsOverall$Heterogeneity
## Tau I^2
## 0.05788512 7.44597151
## Jackson's I^2 Between-cluster heterogeneity
## 65.53000000 3.72000000
## Within-cluster heterogeneity ICC
## 3.72000000 0.50000000
##
## $`4`$aggressionPropensity$resultsOverall$`Proportion of significant results`
## [1] NA
##
## $`4`$aggressionPropensity$resultsOverall$`Publication bias`
## [1] "Publication bias corrections not carried out"
##
##
## $`4`$aggressionPropensity$resultsTable
## k g [95% CI] 95% PI [LB, UB] SE
## 2 -0.03 [-0.9, 0.85] [-0.865, 0.811] 0.07
## Tau I^2
## 0.06 7%Forest plots
Funnel plots
